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Abstract 


We  briefly  go  over  the  well  known  dual  relationship  between 
Dantzig-Wolfe  Decomposition  and  Benders  Decomposition,  in  order  to 
develop  suitable  notation,  and  then  elaborate  upon  the  dual 
relationship  between  nested  versions  of  Dantzig-Wolfe  and  Benders 
Decomposition.  Next  we  develop  a new  pair  of  dually  related  decompo- 
sitions termed  symmetric  Dantzig-Wolfe  and  symmetric  Benders  Decompo- 
sition. Finally  we  discuss  the  advantages  and  disadvantages  of 
applying  nested  and  symmetric  decompositions  to  structured  LP  problems, 
in  particular  to  staircase  structures. 
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DUALLY  EQUIVALENT  DECOMPOSITION  ALGORITHMS 
WITH  APPLICATION  TO  SOLVING  STAIRCASE  STRUCTURES 

by 

L.  Nazareth 

1.  Introduction 

Solving  an  LP  problem  say  P by  the  dual  simplex  method  (Lemke 
[1])  is  equivalent  to  solving  the  dual  of  P by  the  simplex  method 

t 

(Dantzig  [2]).  We  shall  use  the  term  "dually  equivalent"  to  describe 

j 

a relationship  such  as  this,  between  two  linear  programming  algorithms.  \ 

It  is  well  known  that  Benders  decomposition  algorithm  [3]  is 
dually  equivalent  to  the  Dantzig-Wolfe  Decomposition  algorithm  [4], 
see  e.g.,  Lasdon  [5].  Similarly  a nested  version  of  Benders  Decompo- 
sition algorithm  (see  e.g.,  Kallio  [6])  is  dually  equivalent  to  the 
nested  Dantzig-Wolfe  Decomposition  algorithm  (see  Glassey  [7],  Ho  and 
Manne  [8]).  This  paper  is  concerned  with  another  pair  of  dually 
equivalent  algorithms  which  we  believe  may  a)  lead  to  new  and  interest- 
ing ways  to  solve  staircase  systems,  b)  indicate  how  different  optimi- 
zation models  may  be  combined.  Our  paper  is  organized  as  follows: 

1.1 . Overview 

In  Section  2 we  discuss  fundamental  results.  First,  we  discuss 
the  relationship  between  Benders  and  Dantzig-Wolfe  Decomposition, 
primarily  to  develop  notation  and  lay  the  basis  for  the  new  material. 
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Nested  versions  of  Dantzig-Wolfe  Decomposition  have  been  developed  by 
Glassey  [7]  and  Ho  and  Manne  [8].  By  establishing  a dual  equivalence 
with  nested  decomposition,  Kallio  [6]  develops  a nested  version  of 
Benders  algorithm  for  blocking  triangular  matrices,  but  does  not  go 
into  any  detail.  Therefore  we  elaborate  a little  on  this.  Next  we 
develop  what  we  call,  for  want  of  a better  name,  symmetric  Benders 
decomposition  and  symmetric  Dantzig-Wolfe  decomposition.  The  symmetric 
Benders  algorithm  is  related  to  the  tangential  approximation  method  of 
Geoffrion  [9]. 

Finally,  in  Section  3,  we  discuss  solution  strategies  and  the 
advantages  and  disadvantages  of  applying  nested  and  symmetric  decompo- 
sitions to  structured  LP  problems,  in  particular  those  with  staircase 
structures . 

Our  aim  in  this  paper  is  to  discuss  some  basic  approaches  to 
solving  structured  LP  problems.  Implementational  details  will  be  dis- 
cussed at  a later  date. 
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2.  Basic  Results 


where 


We  consider  the  primal  LP  system 


V T 

minimize  I c.  x 
j-1  ~3 


such  that  £ A,  x ) b 


>0 


c and  x.  are  n dimensional  vectors 
"*  j J A 


A is  an  (m  x n ) dimensional  matrix 

j « 


b is  an  m dimensional  vector. 


The  dual  of  (2.1)-P  is 


maximize  it  b 


such  that  Ajir^Cj  j ■ 1 * 2 , ...,n 


ir  > 0 


(2.1)-P 


• (2.2)-D 


where 


tt  is  an  m-dimensional  vector 

Difficulties  of  notation  often  needlessly  complicate  the  descrip- 
tion of  decomposition  algorithms,  particularly  their  nested  versions. 
Therefore  we  shall  confine  ourselves  in  this  section  to  the  LP  systems 
(2.1)-P  and  (2.2)-D  with  n = 2 or  n = 3,  which  is  quite  adequate 
for  our  purpose. 

A summary  of  the  development  of  material  in  this  section  was 
given  in  Section  1.1.  Lasdon  [p.  382,  5],  has  described  the  dual 
equivalence  between  Bender’s  and  Dantzig-Wolfe  decomposition.  In  our 
brief  development  of  nested  versions,  we  shall  parallel  the  development 
in  Lasdon  and  appeal  frequently  to  his  results  and  notation.  Note  that 
nested  algorithms  have  been  introduced  for  specially  structured  LP's  — 
staircase  structures  in  Ho  and  Manne  18],  and  block  triangular 
structures  in  Kallio  [6].  These  are  the  settings  within  which  the 
algorithms  can  be  expected  to  be  useful,  but  they  are  not  restricted 
in  principle  to  these  structures.  Since  our  aim  is  exposition,  we  shall 
let  Aj  be  full  matrices.  Later,  in  Section  3,  we  shall  consider 
special  structures. 

We  shall  also  employ  the  notation 


2.2.  Basic  Decomposition  Algorithms 


2.2.1.  Benders  decomposition  algorithm 

We  first  briefly  describe  Benders  algorithm,  following  Lasdon 
[5],  p.  370].  Consider  the  problem  (2.1)-P  with  n = 2. 


T T 

minimize  c^  x^  + c ^ *2 


such  that  + A2  £2  — - 


*1»*2  - 0 


(2.4)-P2 


The  method,  in  essence,  fixes  x^  at  some  value  and  solves  the  result- 
ing LP  for  X£,  then  adjusts  the  value  of  x^  and  repeats,  a procedure 
known  formally  as  Pripjection,  see  Geoffrion  [10]. 

In  order  to  ensure  feasibility  of  the  projected  LP,  x^  is 
restricted  to  lie  in  the  set  (assumed  non-empty) 

Ri  - {*1  > ola  x2  > ° 
such  that  Aj  ^ x^} 


Then  (2.4)-P2  is  equivalent  to 


min 


-*1 e Hx  I 


X X 

c*  Xj^  + minlc*  *2^A2  -2  — - 
-2 


A1  *1  * *2  - 


(2.5) 
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The  inner  minimization,  if  dualized,  becomes: 
BENDERS  SUBPROBLEM 

maximize  (b  - Aj.  j^)1  u 
such  that  u £ c2 

u > 0 


(2.6)-BSP 


and  thus  (2.5)  becomes: 


min 


inR  j£i  h + "“IQ  - \ u , u e S)i 


2l  - l 


(2.7) 


where 


A T 

s - (u|A2  u _<  c2  , U > 0} 


Let  us  denote  the  extreme  points  of  S by  uP  i - 1 n , 

y 1 •*•»•••»  np  and  the 

extreme  rays  of  S hv  nr  a - i 

y Si*  1 1 » • • • » nr . Lasdon  [5]  establishes 


two  facts 


-F-Ct  1:  The  8et  Rx  is  given  by 


R1  * {*ll \ ij)1  uj  < 0 , i - 1 nr  , xx  > 0} 


where 
r 


are  the  extreme  rays  of  the  set  C £ {u|A*  u < 0 , u > 


0). 
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Fact  2:  If  the  inner  maximization  in  (2.7)  is  unbounded,  then  the  inner 
minimization  in  (2.5)  would  be  infeasible,  contradicting  the  assumption 


that  Thus  in  the  problem  (2.7)  we  need  only  consider  extreme 

points  of  S in  the  inner  maximization. 

Thus  problem  (2.7)  can  be  written  as 


min  fCT  x1  + max[(b  - A1  x..)T  u^  , i = 1,  2,  ....  n ]j 
*1  G Rl(  - 1 1 -i  P ) 


(2.8) 


and  this  is  equivalent  to 


minimize  z 


T T p 

such  that  z 2 c^  Xj  + (b  - x^)  u^  , i **  1,  2, 


0 > (b  - Aj_  x^1  u*  , 


2,  • • • » n 


xx  > 0 


A T 

Writing  v - z - c^  x^  this  becomes 
BENDERS  MASTER 

T 

minimize  c^  x^  + v 


such  that  v 2l  ^ ^ x^)  i ■ 1,  2, 


0 > (b  - Al  x1)T  uj  i - 1,  2,  .. 


(2.9)-BM 


x^  > 0 
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In  practice,  a relaxation  strategy  is  used,  so  that  only  a small 
fraction  of  the  constraints  are  employed  in  a restricted  master.  Again 


see  Lasdon  [5,  p.  375],  for  details. 


2.2.2.  Dantzig-Wolfe  decomposition 

We  now  work  with  the  dual  of  (2.4)-P2 


maximize  b it 


such  that  ir  c^ 


A2  I — -2 


(2.10)-D2 


TT  > 0 


Treating  the  constraints  A2  I - c2’  71  - ® as  t*le  subproblem. 


we  have 


" “ l x,  + I ^1  * 

j-1  1 J j-1  2 2 

n 

rP 

I x*  - 1 • 

j-i  J 


The  Dantzig-Wolfe  [4]  master  problem  becomes 


DANTZIG-WOLFE  MASTER 


ft 

f i 


maximize  £ (bT  u^)X  + \ (b1  u*)jj. 

j-1  ' "-1  J j-1  " J 


T rN 


such 


that  f (A*  uP)A  + l (A1  uj)p  < c. 
j-1  1 J J j-i  1 "J  J 1 


T rv 


(2.11)-DWM 


! v1.  K . U,  > 0 

j-1  J J J 


Extreme  points  are,  of  course,  developed  as  needed  by  a restricted 
master.  Let  x^  be  the  dual  variables  for  the  first  set  of  constraints, 
and  v the  dual  variable  for  the  convexity  constraint.  Then  the 
Dantzig-Wolfe  subproblem  becomes 


maximize  (b  - A^  x^)  u 


such  that  u £ 


• (2 .12)-DWSP 


u > 0 


2.2.3.  Dual  equivalence  of  Benders  and  Dantzig-Wolfe  decomposition 

This  is  easily  seen  by  comparing  (2.6)-BSP  and  (2.12)-DWSP,  these 
being  the  subproblems  corresponding  to  the  two  decompositions,  and  noting 
that  (2.9)-BM  is  the  dual  of  (2.11)-DWM.  It  can  also  be  shown  that 
both  algorithms  employ  the  same  test  for  optimality. 


'■TT'r 


* - 


2.3.  Nested  Decomposition  Algorithms 
2.3.1.  Nested  Bender's  decomposition 

Let  us  now  consider  the  system  (2.1)-P  with  n = 3.  Using  the 
notation  (2.3)-N,  given  at  the  end  of  Section  2.1,  this  can  be  written 
in  the  form 

T T 

minimize  &2  *2  + c.^  x^ 
such  that  A2  *2  + A3  ^3  — b 

*2»  £3  2 0 

Then  following  the  development  of  Section  2.2.1  we  can  apply 
Benders  Decomposition  to  (2.13)-P3  and  we  have  the  subproblem 
NESTED  BENDER  SUBPROBLEM  1 

~ T 

maximize  (b  - A 2 x2)  u^  \ 

X f 

such  that  A3  ^ < c,  >(2 .14)-NBSP1 

b?l  — 0 / 

where  a subscript  on  u is  introduced  to  distinguish  variables  at  the 
first  nested  level. 


f 


Analogously  to  (2.9)-BM  of  Section  2.2.1,  the  first  level 


Benders  Master  Is 


NESTED  BENDER  MASTER  1 


minimize  C2  *2  + V1 


such  that  2 (b  — A 2 u^  1*1,  2,  ....  n^ 


° 2 (b  " A2  *2)  u1±  1 - 1,  2,  ....  n} 


*2  - 0 


where  u^,  are  defined  in  an  analogous  way  to  the  extreme  points 

in  (2.9)-BM. 


Then  if  we  let 


■ (2 . 15)-NBM1 


u 1 " Infr  uj2,  ....  «Jn  ] 


,.r  r r r r 

U1  f-ll’  -12 *hn  1 


and  recalling  the  definitions  of  A2  and  *2  we  can  write 

(2.15)-NBM1  as 


i-f  , * . • \\ 


minimize 


i 

T t T 

£1  5l  + | *2  252  + V1 


T { T 

such  that  (U^  A^)^  + j(u^  A2)x2  + ~ V1 

! 

i 

T ' T 

(Uj  A^+jVuJ  A2)x2 


> uj  b 


T 

>uj  b 


-1’  -2  — ® 


where  e ■ (1,  1,  1) 


If  we  now  apply  Benders  Decomposition  to  this  system,  and  let 
the  subproblem  correspond  to  the  variables  (x2>  v^) , we  have  the  second 
level  Benders  master  given  by 


NESTED  BENDER  MASTER  2 
T 

minimize  c^  x^  + v2 


,n2p 


f (2.16)- 

NBM2 


,n2r 


~1  ~ ° J 
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H3E 


'.’m 


IBHQ  MUM| 


■*> 


and  the  subproblem  Is  given  by 


and  this  can  be  rewritten  as 


' 


maximize  [(bT  uj  bT  U*)  - x*(A J uj  A*  u[)]u2 


such  that 


Uj  > 0 


(2 . 17)-NBSP2 


i 
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2.3.2.  Nested  Dantzig-Wolfe  decomposition 
Now  consider  the  dual  of  (2.13)-P3 


maximize  b it 


such  that  A^  jt  ^ 


T 

A3  2 — £3 


it  > 0 


■ (2 . 18) -D3 


Then  carry  out  the  standard  Dantzig-Wolfe  decomposition  cf  §2.2.2 
we  have  the  master  given,  analogously  to  (2.11)-DWM  by 


NESTED  DANTZIG-WOLFE  MASTER  1 


ni  ni 

lp  T n lr  T r 

maximize  [ (b  up  )*_  + £ (b  uf.)p 


L - -li  li  L 
i-1  i-1 


li  li 


ni  ni 

lp  ^ lr 

such  that  l (A2  ul^Aj^  + l (a!|  iJi±)  Vi1±  > 
i-1  i-1 


Xli*  pli  > 0 


(2.19)-NDWM1 


> --  , 


r.  > . 


where  u^,  i ■ 1*  •••»  n^  and  u!^,  i ■ 1*  •••»  n^r  are  the 
extreme  points  and  extreme  rays  of  the  subproblem  which  is  given  by 
NESTED  DANTZIG-WOLFE  SUBPROBLEM  1 

- T 

maximize  (b  - Aj  *2^  E 

such  that  A^  it  c3  (2.20)-NDWSPl 

TT  _>  0 

where  *2  are  the  dual  variables  of  rows  other  than  the  convexity 
row  in  (2.19).  If  we  substitute  for  and  in  (2.19)-NDWM1 

we  get 

maximize  \ (bT  \ (bT  11^)11^ 


Writing 


and 


and 


'll’ 

this  becomes 

NESTED  DANTZIG-WOLFE  MASTER  1 


U1  “ (-ll*  -In.  } 

Ip 


Uj  - (u[  , ....  ) 

1 r 


T T 

-1  " (X11»  Aln,  } 

IP 


T T 

H " (Vii*  •••>  V,  ) 


maximize  (bT  U^)X^  + (bT 


such  that  (A^  U^)A^  + (A^  <. 


I (a*  uJ)ax  + (a*  uJ)Hl  < c2 

i 
< 
i 
i 


! T 

! s Ai 


- i 


i 


i -i*  *i 


> o 


f(2.21)-NDWMl 
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Now  if  we  repeat  the  Dantzig-Wolfe  decomposition,  letting 
the  subproblem  correspond  to  the  boxed  set  of  constraints,  we  obtain 
the  second  level  master  as 

NESTED  DANTZIG-WOLFE  MASTER  2 > 


2p  1 2r  T i 

maximize  £ ( <bTuJj b U*>-2i] *21  + £ Ub^jJj  b^u^Ju^ 

1*1  a j 


% T nl  T r n "2r 

V r /"A^IlPl  A^ITr\.,P  a.  V 


such  that  ['  [(AjuP|AjuJ)uP1]X21  + £ [ (aJuJ|  A*U]y u^Jp^  < (^ 

1-1  ! i-1  ! 


•(2.22)-NDWM2 


> 

1 *2i 

i-1 


*2i’  p2i  - 0 


If  we  denote  the  dual  variables  corresponding  to  the  first  set  of 
T 

constraints,  by  x^,  our  subproblem  is  then  given  by 


maximize  [(bT  u£  j bT  U^)  - x*(A*  u£  i A^  u[)]u2 


►(2.23)-NDWSP2 


such  that 


TtT  S2iW 


2.3.3.  Dual  equivalence  of  nested  Benders  decomposition  and  nested 


Dantzig-Wolfe  decomposition 

If  we  denote  the  dual  variables  of  (2 .19)-NDWM1  by  Xj  and  v^ 
then  we  can  see  that  (2.15)-NBM1  is  the  dual  of  (2.19)-NDWM1.  Also 
(2.14)-NBSP1  is  the  same  as  (2 . 20)-NDWSPl . Again,  if  we  compare 
(2.16)-NBM2  with  (2.22)-NDWM2  and  let  and  be  the  dual 

variables  of  (2.22)-NDWM2,  we  see  that  (2.22)-NDWM2  and  (2.16)-NBM2 
are  duals.  Also  (2.17)-NBSP2  is  the  same  as  (2.23)-NDWSP2.  Note 
again  that  we  will  deal  in  practice  with  restricted  master  problems. 


2.4.  Symmetric  Decomposition  Algorithms 
2.4.1.  Symmetric  Benders  decomposition 

Let  us  return  to  problem  (2.4)-P2  of  Section  1.1 


T T 

minimize  c^  ^ 


such  that  A^  x^  + x^  b 


-1’  2 2 — 0 


(2.24)-P2 


We  shall  employ  the  following  notation  of  Geoff rion  19] 

G Km|a  such  that  A^  x^  0} 

for  i - 1,  2. 
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A T 

1(yi)  “ min  {c^  xjAj^ 


i - 


If  the  set  is  infeasible  for  the  given  y^  then  v^^)  - + 
Geoffrion  [9]  shows  that  (2.24)-P2  is  equivalent  to 

minimize  v^^)  + v^^) 


such  that  + ^2  — - 


(2.25)-P2 


i - 1,  2 


Fact  3:  In  an  essentially  identical  proof  to  that  used  by  Lasdon 
[3]  to  establish  Fact  1,  Section  2.2.1,  we  have 


{Xi  G Yi  " 2i  H1;)  < 0 • j - 1,  2,  ....  nlr> 


for  i ■ 1,  2,  where  u^j,  j ■ 1,  2,  ...»  nir  are  the  extreme  rays 
of  the  sets  - {u|A^  u £ 0,  u 0},  i ■ 1,  2.  □ 


- >:  -vf  v.  . • • > * 


Consider  now  the  objective  function  of  (2.25)-P2 


T T 

vl(2l)  + v2(x2)  = min_  {£l  x1|A1  ^ > j^)  + min^  {c2  x2U2  x2  > 


If  we  dualize  each  of  these  minimizations,  we  can  write  this  as 


v1(^1)  + v2(y2)  = max  {y^  u^lA^  _<  c^}  + max  {y2  u^A*  u2  £2> 


T i .T 


ux  > 0 


u2  > ° 


Fact  4:  Again,  analogously  to  Fact  2 of  Section  2.2.1,  if  either 
maximization  problem  above  went  to  + <»  along  an  extreme  ray,  the 
corresponding  dual  minimization  problem  would  be  Infeasible.  This 
would  contradict  the  constraint  y±  S Therefore  we  need  only 

consider  extreme  points  of  the  feasible  sets 


^ -i. I Ai  -i  - -i  * -i  - » 


i - 1,  2 . 


Let  us  denote  these  extreme  points  by  -ij’  j “ lf  2 % 

for  i * 1,  2. 

Then  (2.25)-P2  can  be  written  as 

min  [nax{^  u^,  , j - 1,  ...,  n^  } + max{£2  ui|  , j - 1,  2,  ....  n2p}] 

IV  *2 


such  that  + ^2  — - 


6 Yi  « t-1.2 


V i*i**r'*t2  • 


• * 'f  < • 


Thus,  finally,  this  can  be  written  as  the  symmetric  Bender  Master 
problem 


SYMMETRIC  BENDER  MASTER 


minimize  + z^ 


such  that  z,  > y,  u? 


1 * -UI j 


j - 1,  ...,  n 


lp 


0 > 


T r 


1 *1  Sij 


J " • • • , n. 


z2  — 


0 > 


lr 


22  Hy  • i “ 1.  •••.  n2p 


^2  -2j  » J “ 1 n2r 


t < 2i  + 22 


> (2.26)-SBM 


and  the  symnetrlc  Bender  subproblems  are  given  by 
SYMMETRIC  BENDER  SUBPROBLEM  i 

T 

maximize  ^ 


such  that  A*  c^ 


(2.27)-SBSPi 


for  i - 1,  2. 


ut  > 0 


21 


-T-— ”■  , ■■■  ■ 


Note  that  the  master  has 
in  practice,  we  shall  deal  with 


block  angular  form,  and  that  again, 
a restricted  master  problem. 


2.4.2.  Symmetric  Dantzig-Wolfe  decomposition 
Consider  again  the  problem  (2.10)-D2 

T 

maximize  b it 

T 

such  that  tt  c^ 

T 

A2  - — -2 

ir  > 0 


> (2.28)-D2 


A feasible  solution  tt^  of  the  set 
satisfies 


E i.  £i  • 


it  > 0} 


li- 

lp  lr 

2l  “ Xlj  Hlj  + ^ plj  Hy 


with 


ii, 

ip 

I * 1 

j»l  J 


X lj  - 0 ’ plj  - 0 


22 


tamte&i — 

♦ ■ r f 


If  we  denote  the  dual  variables  corresponding  to  the  first 
two  sets  of  constraints  by  ^ and  y2,  then  the  two  subproblems 
l for  syiumetric  Dantzig-Wolfe  Decomposition  are  given  by 

SYMMETRIC  DANTZIG-WOLFE  SUBPROBLEM  i 

l 

maximize  u^ 

j 

such  that  < c±  (2. 30)-SDWSPi 

u±  > 0 

2.4.3.  Dual  equivalence  of  symmetric  Benders  and  symmetric  Dantzig- 
Wolfe  decomposition 

This  is  immediately  seen  by  noting  that  (2.27)-SBSPi  and 
(2 . 30)-SDWSPi  are  identical,  and  that  (2.26)-SBM  is  dual  to  (2.29)-SDWM. 

Again,  in  practice  we  use  restricted  master  problems. 


3.  Concluding  Remarks  on  Solution  Strategies  and  Application  to 
Special  Structures 

In  Section  2 we  dealt  with  algorithms  for  solving  (2.1)-P 
w and  (2.2)-D,  confining  the  discussion  to  the  case  n » 2 or  3. 

It  is  easy  to  see  how  the  algorithms  extend  to  arbitrary  n.  In  this 
section  we  elaborate  upon  these  algorithms  discussing,  in  particular, 
their  advantages  and  disadvantages.  We  shall  be  primarily  concerned 
with  their  application  to  staircase  structures.  Figure  3.1a)  and 
3.1b)  illustrate  two  dual  staircase  LP's,  which  are  special  cases 
of  general  LP's  illustrated  in  Figures  3.2a)  and  3.2b) 


Note  that  in  staircase  structures  there  are  often  only  a 
few  linking  columns  in  Figure  3.1a)  and  correspondingly  only  a few 
linking  rows  in  the  LP  dual  to  it.  Figure  3.1b). 

3.1.  Nested  Decomposition  Algorithms 

The  algorithms  for  Glassey  [7]  and  Ho  and  Manne  [8]  were 
developed  for  staircase  LP’s  of  the  form  shown  in  Figure  3.1b). 

The  algorithm  of  Kallio  [6]  is  a generalization  of  these  to  the  case 
when  the  matrix  is  block  upper  triangular. 

In  these  algorithms,  stage  4 would  correspond  to  the  first 
subproblem  and  stages  1,  2,  and  3 to  the  first  level  master.  Then 
stage  3 of  this  master  is  the  second  level  subproblem  and  stages  1 
and  2 the  second  level  master,  and  so  on.  A major  contribution  of  Ho 
[11]  was  to  develop  a workable  implementation  of  the  procedure  and 
demonstrate  its  effectiveness, since  many  details  must  be  worked  out 
in  order  to  specify  a workable  nested  decomposition  method. 

We  should  note  the  following: 

a)  The  nested  decomposition  method  of  Section  2.3.2  works  best  for 
structures  of  the  form  3.1b),  since  each  master  will  then  have 
relatively  few  rows.  If  applied  to  the  staircase  structure 
3.1a),  where  each  stage  has  many  more  rows  than  columns,  each 
master  and  thus  subsequent  subproblems  would  become  very  large. 

b)  All  structure  within  each  stage  can  be  lost  in  the  nested  Dantzlg 
Wolfe  Decomposition.  Thus,  if  the  constraints  of  some  stage  are 
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Implicitly  defined,  e.g.,  by  a network,  we  cannot  exploit  this 
added  structure  in  the  problem. 


i 


c)  The  procedure  lacks  symmetry  with  respect  to  interchange  of  stages. 


i 

? 


d)  There  is  an  Inherent  propogatlon  and  build  up  of  error  in  the 
nested  decomposition  process.  Errors  in  the  extreme  points  of  a 
subproblem,  result  in  errors  in  the  columns  of  the  master  to 
which  these  extreme  points  are  transmitted,  which  in  turn  result 
in  errors  in  the  next  level  of  the  decomposition.  For  a more 
detailed  discussion  see  Nazareth  [ 1 ] 2 . 

e)  A post-optimality  procedure  (called  Phase  3)  is  needed  to  recover 
the  solution  (see  Ho  [11]). 

f)  Finally,  note  that  the  nested  Dantzig-Wolfe  decomposition  can  be 
carried  out  in  other  ways.  Thus  in  Figure  3.1b)  stages  3 and  4 
could  be  the  first  level  subproblem  and  stages  1 and  2 the  first 
level  master  then  each  of  these  could  be  decomposed  in  turn  into 
a master  and  subproblem.  A serious  disadvantage  of  such  a pro- 
cedure is  that  a Phase  3 process  would  be  model  at  Intermediate 
stages  in  order  to  recover  an  extreme  point  of  a subproblem,  for 
transmission  to  a master. 

By  the  dual  equivalence  discussed  in  Section  2.3.3,  we  would 
expect  the  Nested  Benders  algorithm  to  work  best  on  structures  of  the 
form  3.1a)  and  3.2a).  Because  we  wish  to  have  primal  feasibility 
explicitly  in  3.2b)  we  might  consider  applying  nested  Benders  decompo- 
sition to  this  structure.  However,  note  that  we  would  then  have  the 


f 

disadvantages  of  nested  Dantzig-Wolfe  decomposition  applied  to  3.2a) 

I 

i.e.,  at  first  sight  it  appears  as  though  we  get  master  problems 
with  a large  number  of  rows,  though  there  may  be  ways  to  get  around 
this  by  working  with  the  dual. 

3.2.  Symmetric  Decomposition  Algorithms 

In  contrast  with  the  Nested  Dantzig-Wolfe  decomposition  algorithm, 
the  symmetric  Dantzig-Wolfe  algorithm  applies  most  naturally  to  struc- 
tures of  the  form  3.1a)  and  3.2a),  since  the  size  of  the  symmetric 
, Dantzig-Wolfe  master  is  determined  by  the  number  of  columns.  Let 

denote  the  number  of  linking  columns  for  stage  i in  Figure  3.1a) 
and  r^  the  number  of  remaining  columns  for  stage  i (correspondingly 
and  r^  are  the  number  linking  rows  and  remaining  rows  in  Figure 

3.1b)).  If  we  decompose  into  4 subproblems,  one  for  each  stage  in 

A 3 

Figure  3.1a),  the  symmetric  Dantzig-Wolfe  master  has  ri  + ^i»l  ^1  + ^ 

rows  and  a structure  illustrated  in  Figure  3.3.  Note  that  the  shaded 

blocks  correspond  to  dense  matrices,  a potential  disadvantage. 

r4  r3 

The  first  portion  of  the  matrix  has  £j_  r^  + ^ columns. 

In  general,  if  extreme  points  of  subproblems  are  not  degenerate,  and 
cancellation  does  not  occur,  we  would  expect  that  variables  correspond- 
ing to  this  first  part  to  have  non-zero  values.  Thus  we  would  expect 
a basic  solution  of  the  symmetric  master  to  have  approximately 

l.  + 4 columns  from  the  dense  block  diagonal  part  of  the  matrix. 

If  is  small,  these  would  be  relatively  few.  Operations  involving 
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a basis,  for  example,  LU  factorization,  in  this  system  can  then  be 
handled  more  efficiently  than  corresponding  operations  in  the  sparse 
staircase  system  3.1b). 


rl+*l  r2+*7  r3+A4  r4+<,4 


In  contrast  to  the  nested  algorithms  note: 


a)  No  Phase  3 is  required  to  recover  the  solution. 

b)  Special  structure  within  a subproblem  is  not  lost.  In  particular 
different  models  can  be  integrated  into  a single  model  and  at  the 
same  time  the  special  structure  of  each  model  can  be  exploited. 

c)  There  is  no  transmission  of  error  from  stage  to  stage. 

d)  It  is  also  possible  to  do  symmetric  decomposition  in  a nested 
manner.  For  example,  we  could  treat  stages  1 and  2 and  stages 

3 and  4 as  the  first  level  subproblems  and  obtain  a master  with 

3 A 3 A 

II  + ^2  ri  + *"2  + ^ rows  an<*  ^1  + ri  c°lunins  in  the 

front  portion  of  the  matrix.  The  procedure  can  then  be  repeated 
within  each  pair  of  stages.  Note  that  in  this  case  there  is  trans- 
mission of  error  from  our  level  of  decomposition  to  all  subsequent 
levels. 

Finally  by  the  dual  equivalence  of  Section  2.4.3,  the  above 
discussion  applies  to  the  Symmetric  Benders  algorithms,  which  is  used 
on  structures  of  the  form  3.1b). 

In  effect  in  symmetric  decomposition,  we  are  exchanging  our  LP 
problem  for  a block/dual  block  angular  structure  of  special  form  of 
Figure  3.3  with  dense  blocks  and  smaller  subproblems.  The  latter  do 
not  lose  any  structures  of  their  own  and  are  amenable  to  solution  by 
specialized  techniques.  Symmetric  decomposition  may  have  advantages 
for  solving  certain  sorts  of  staircase  models  and  for  combining  different 
models  into  a single  integrated  LP  model. 
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